Homo economicus in visual search

How do reward outcomes affect early visual performance? Previous studies found a suboptimal influence, but they ignored the non-linearity in how subjects perceived the reward outcomes. In contrast, we find that when the non-linearity is accounted for, humans behave optimally and maximize expected reward. Our subjects were asked to detect the presence of a familiar target object in a cluttered scene. They were rewarded according to their performance. We systematically varied the target frequency and the reward/penalty policy for detecting/missing the targets. We find that 1) decreasing the target frequency will decrease the detection rates, in accordance with the literature. 2) Contrary to previous studies, increasing the target detection rewards will compensate for target rarity and restore detection performance. 3) A quantitative model based on reward maximization accurately predicts human detection behavior in all target frequency and reward conditions; thus, reward schemes can be designed to obtain desired detection rates for rare targets. 4) Subjects quickly learn the optimal decision strategy; we propose a neurally plausible model that exhibits the same properties. Potential applications include designing reward schemes to improve detection of life-critical, rare targets (e.g., cancers in medical images)

Predicting response time and error rates in visual search

A model of human visual search is proposed. It predicts both response time (RT) and error rates (RT) as a function of image parameters such as target contrast and clutter. The model is an ideal observer, in that it optimizes the Bayes ratio of target present vs target absent. The ratio is computed on the firing pattern of V1/V2 neurons, modeled by Poisson distributions. The optimal mechanism for integrating information over time is shown to be a ‘soft max’ of diffusions, computed over the visual field by ‘hypercolumns’ of neurons that share the same receptive field and have different response properties to image features. An approximation of the optimal Bayesian observer, based on integrating local decisions, rather than diffusions, is also derived; it is shown experimentally to produce very similar predictions to the optimal observer in common psychophysics conditions. A psychophyisics experiment is proposed that may discriminate between which mechanism is used in the human brain

Turning off or dimming a device screen based on user attention

Device screens are often set to turn off and/or dim automatically if no user interaction is detected for a specified amount of time. Turning off or dimming the screen saves power and prolongs the amount of time the device can operate without needing to recharge the battery. However, such timeout-based actions can result in false positives or negatives. With user permission, this disclosure utilizes contextual input of a user’s gaze and attention for management of the automatic turn off or dimming of the device screen. The techniques are applied to reduce the false positives and negatives and ensure that the screen stays on longer if the user is still engaged with the device and turns off or dims before the timeout if the user has stopped using the screen

Operator Drowsiness Test

This publication details a quantifiable and objective operator drowsiness test. The test takes between 30 seconds to two (2) minutes to be administered. Any smartphone that has a front-facing camera and the supporting software can run the newly-developed and self-administrable test. It leverages years in sleep deprivation research that have found objective correlations between drowsiness (or alertness) and physical and behavioral parameters, such as: gazing, facial features, pupil size, blink rate, blink duration, breathing, pulse, head movements, face skin-tone, speech pattern, and vocal sound. In addition, the mass use of smartphones with rear-facing and front-facing cameras gives researchers the opportunity to deploy this new operator drowsiness test to a wide audience

Differentially Private Heatmaps

We consider the task of producing heatmaps from users' aggregated data while protecting their privacy. We give a differentially private (DP) algorithm for this task and demonstrate its advantages over previous algorithms on real-world datasets. Our core algorithmic primitive is a DP procedure that takes in a set of distributions and produces an output that is close in Earth Mover's Distance to the average of the inputs. We prove theoretical bounds on the error of our algorithm under a certain sparsity assumption and that these are near-optimal.Comment: To appear in AAAI 202

A mathematical framework for the design and analysis of feature biasing strategies

Please see attached pdf file

An integrated model of top-down and bottom-up attention for optimal object detection

Integration of goal-driven, top-down attention and image-driven, bottom-up attention is crucial for visual search. Yet, previous research has mostly focused on models that are purely top-down or bottom-up. Here, we propose a new model that combines both. The bottom-up component computes the visual salience of scene locations in different feature maps extracted at multiple spatial scales. The topdown component uses accumulated statistical knowledge of the visual features of the desired search target and background clutter, to optimally tune the bottom-up maps such that target detection speed is maximized. Testing on 750 artificial and natural scenes shows that the model’s predictions are consistent with a large body of available literature on human psychophysics of visual search. These results suggest that our model may provide good approximation of how humans combine bottom-up and top-down cues such as to optimize target detection speed. 1

A goal oriented attention guidance model

Abstract. Previous experiments have shown that human attention is influenced by high level task demands. In this paper, we propose an architecture to estimate the task-relevance of attended locations in a scene. We maintain a task graph and compute relevance of fixations using an ontology that contains a description of real world entities and their relationships. Our model guides attention according to a topographic attention guidance map that encodes the bottom-up salience and task-relevance of all locations in the scene. We have demonstrated that our model detects entities that are salient and relevant to the task even on natural cluttered scenes and arbitrary tasks.